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Statement  of  Problem 

Visual  information  processing  appears  to  operate  in  two  relatively  distinct  stages 
(Hoffman,  1978;  1979;  Treisman,  1988).  The  initial  stage  is  a  parallel,  feature 
extraction  process  that  produces  separate  retino-toplc  maps  corresponding  to  each  of 
several  dimensions  such  as  color,  orientation,  motion,  depth,  etc.  A  second  stage  of 
focal  attention  is  responsible  for  integrating  the  separate  features  into  "object 
representations".  This  stage  is  slow  and  serial. 

Previous  research  funded  by  ARO  (DAAG29-83-K-0049)  explored  the  possibility 
that  extensive  practice  in  visual  detection  and  search  tasks  could  result  in  automatic 
detection  of  targets  that  would  no  longer  require  the  the  use  of  limited-capacity 
attentional  resources.  We  found  that  such  training  did  dramatically  enhance  these 
skills  but  that  limited-capacity  processes  continued  to  set  limits  in  dual-task 
situations  in  which  the  observer  was  required  to  monitor  several  sources  of  visual 
information.  These  inferences  were  verified  by  simultaneously  monitoring  behavioral 
performance  measures  as  well  as  components  of  the  event-related  brain  potential  (ERP) 
that  are  known  to  reflect  attentional  processes.  Much  of  this  work  is  summarized  in 
Hoffman  (1986)  and  Hoffman  (in  press). 

The  current  work  was  directed  at  gaining  a  further  understanding  of  spatial 
attention  and  its  role  in  perceptual  skills.  An  additional  project  was  conducted  in  the 
area  of  language  and  ERPs. 


Summary  of  Results 

Eye-movements  and  attention.  Spatial  attention  can  be  allocated  to  different 
areas  of  the  visual  field  within  a  single  eye-fixation,  showing  that  there  is  some  degree 
of  independence  between  the  systems  controlling  attention  and  the  eyes.  Previous 
investigators  have  explored  the  degree  of  independence  by  requiring  subjects  to 
simultaneously  attend  to  different  points  in  the  field  field  and  move  their  eyes  to  the 
same  or  different  locations.  For  example,  Klein  (1979)  reasoned  that  if  the  two  systems 
are  independent,  then  subjects  should  be  able  to  move  to  their  eyes  to  one  location  and 
attend  to  another  without  any  mutual  interference.  Although  he  reports  results 
consistent  with  this  hypothesis,  there  are  methodological  problems  with  his  study  that 
make  any  conclusions  suspect. 

Baskaran  Subramanian  in  his  master's  thesis  investigated  this  question  with  an 
Improved  methodology.  In  his  first  experiment,  subjects  were  instructed  to  make  a 
saccade  to  a  particular  location  in  the  visual  field.  Unlike  previous  research,  the 
saccade  signal  was  auditory  rather  than  visual,  thus  precluding  the  problem  of 
automatic  capture  of  attention  by  a  visual  t  r-'ent.  Shortly  before  the  eyes  began  to 
move,  a  visual  array  was  briefly  presented  a  id  tV  subject  was  required  to  determine 
which  of  two  letters  was  present.  Subramam.-  mnd  that  letters  were  more  likely  to 
be  discriminated  in  the  same  location  as  the  target  of  the  saccade  showing  that  subjects 
were  allocating  attention  to  this  area  as  part  of  the  saccade-programming  process. 

In  a  second  experiment,  he  investigated  whether  this  relationship  between 
attention  and  eye-movements  was  obligatory.  Subjects  made  a  saccade  to  a  particular 
location  over  a  long  series  of  trials.  In  addition,  subjects  were  provided,  prior  to  each 
trial,  the  probable  location  of  a  target  letter.  In  the  absence  of  eye-movements,  subjects 
attended  to  this  cued  location  and  gained  benefits  when  the  target  was  presented  in  that 
location  and  suffered  costs  when  it  appeared  in  non-cued  locations.  When  eye- 
movements  were  required,  however,  there  were  no  costs  and  benefits  associated  with 
the  cue;  instead  targets  were  detected  best  in  the  direction  of  the  saccade  even  when 
another  area  was  the  likely  location  of  the  target.  Thus,  selective  attention  appears  to 
be  a  necessary  component  of  saccadic  eye-movements  and  probably  is  responsible  for 
providing  the  spatial  coordinates  to  the  controlling  mechanism. 
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Attention  and  Objects.  Is  attention  allocated  to  spatial  positions  or  to  "objects"? 
Our  phenomenlogical  experience  suggests  that  it  is  objects  and  research  by  John 
Duncan  (1984)  supports  lhat  claim.  Duncan  examined  people’s  ability  to  divide 
attention  between  separate  features  that  either  belonged  to  the  same  or  different 
objects.  Divided  attention  performance  was  superior  in  the  same  object  condition. 
Unfortunately  features  also  tended  to  be  closer  together  in  this  condition  so  the  results 
are  also  consistent  with  the  claim  that  attention  is  allocated  as  a  "spotlight"  to  spatial 
areas. 


Sharon  Zerlin  in  her  master's  thesis  designed  an  experiment  to  disentangle 
spatial  proximity  and  objeclness.  In  addition,  she  sought  to  use  a  more  convincing 
method  of  defining  what  constitutes  an  object.  She  had  subjects  identify  a  letter  that 
was  on  a  moving  surface.  "Noise"  letters  could  appear  on  the  same  surface  (same  object 
condition)  or  a  different  surface  moving  in  the  opposite  direction  (different  object 
condition).  In  addition,  the  noise  letters  could  appear  either  close  to  or  far  from  the 
target  letter.  The  degree  of  interference  caused  by  the  noise  was  used  as  the  measure  of 
atlentional  allocation.  The  results  were  clear.  Both  spatial  proximity  and  objectness 
were  independent  determinants  of  attention.  Features  on  the  same  object  and  close  to 
the  target  are  processed  more  than  features  further  away  and  /or  on  a  different  object. 

Temporal  Integration  and  saccades.  In  pilot  work  for  her  doctoral  dissertation. 
Sharon  Zerlin  has  been  studying  the  process  by  which  we  integrate  information  across 
fixations.  It  seems  likely  that  there  is  a  visual  memory  that  allows  information  from 
the  previous  fixation  to  be  integrated  with  information  from  the  current  fixation.  Is 
this  memory  limited  in  capacity  and  what  are  its  decay  characteristics?  Zerlin 
modified  the  partial  report  procedure  of  Sperling  (1960)  to  investigate  this.  Subjects  see 
a  rectangular  array  of  letters  in  one  fixation.  Following  an  eye  movement  to  a  new 
fixation  point,  they  receive  a  visual  cue  indicating  one  letter  for  report.  We  have  found 
that  subjects  can  reliably  report  the  cued  letter  at  short  delays  of  the  cue  but  that 
performance  deteriorates  rapidly  with  increases  in  cue  delay.  Comparable 
performance  is  obtained  in  a  fixation  condition  when  no  changes  in  eye  position  are 
allowed.  This  preliminaiy  result  suggests  that  there  Is  a  rapidly  decaying  memory 
representation  that  allows  for  the  integration  of  information  across  saccades.  In 
addition,  it  appears  that  this  memory  is  defined  in  spatlo-topic  coordinates  rather  than 
retinal  coordinates  (subjects  subjectively  see  the  bar  over  the  position  of  the  letter  lhat 
appeared  in  the  bar’s  screen  position,  rather  than  Its  retinal  position).  Further 
experiments  will  be  required  to  examine  the  question  of  capacity  limits. 

N400  and  Semantic  Congruity.  Artl  Nigam’s  master’s  thesis  investigated  the 
relation  between  the  N400  component  of  the  ERP  and  semantic  anomaly.  Previous 
research  by  Kutas  and  Hillyard  (1982)  found  that  a  large  negative  component  with  a 
latency  of  about  400  msec  occurred  in  the  ERP  when  subjects  encountered  a 
semantically  anomalous  word.  For  example,  the  sentence  frame:  "lie  likes  cream  and 

sugar  in  his _ "  followed  by  the  word  "socks"  would  elicit  an  N400.  This  N400  could 

be  tied  to  any  of  several  different  levels  in  language  representation.  It  cou’  a  index  the 
surface  level  of  the  word  itself  or  it  could  be  associated  with  the  "deep"  o  semantic  level 
of  meaning. 

Nigam  evaluated  these  possibilities  by  comparing  the  usual  N400  situation  with 
one  in  which  the  last  word  Is  replaced  by  a  picture  representing  the  same  concept  (eg.  a 
line  drawing  of  socks).  Pictures  and  words  have  very  different  surface  representations 
but  can  share  the  same  underlying  concept.  Thus,  if  the  pict  ure  condition  produces  an 
N400  with  the  same  latency,  amplitude,  and  scalp  distribution  as  the  word  condition, 
one  could  conclude  that  the  N400  is  indexing  processing  at  a  semantic  level.  This  is 
exactly  what  she  found.  It  should  be  noted  that  subjects  could  not  not  have  simply  been 
naming  the  pictures  and  responding  on  the  basis  of  the  name  because  picture  naming  is 
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about  200  msec  slower  than  word  naming  and  this  would  have  produced  a  dilTerence  In 
latency  between  the  two  conditions.  No  such  latency  shift  was  observed. 

Summary 

We  found  that  visual  attention  is  not  only  important  in  search  and  detection  tasks 
but  plays  a  role  in  the  control  of  action  systems  such  as  eye-movements.  Subjects  must 
attend  to  the  spatial  position  that  is  the  target  of  the  eye-movement.  We  believe  that 
that  this  process  is  important  for  two  reasons.  One  is  that  saccades  are  not  randomly 
distributed  in  a  scene  but  tend  to  be  concentrated  on  areas  of  high  Interest  or 
information  density.  They  are  guided  by  the  information  acquired  through  spatial 
attention.  Second,  attending  to  the  target  area  of  a  saccade  may  produce  a  visual 
memory  representation  of  that  area  which  can  be  compared  with  the  information 
available  at  the  fovea  after  the  eye-movement  has  been  executed.  These  should  match 
and  a  failure  to  do  so  would  constitute  evidence  that  the  scene  had  moved  in  addition  to 
the  eye.  Thus,  this  mechanism  could  provide  part  of  the  Information  required  for 
perceptual  stability  in  the  face  of  eye-movements. 

Attention  can  be  guided  not  only  by  the  spatial  coordinates  available  to  the  visual 
system  but  also  by  the  structure  of  the  surfaces  and  objects  that  are  visible.  We  found 
that  both  of  these  are  important  in  the  allocation  of  attention.  It  still  remains  to  be 
determined  what  constitutes  an  object  to  the  attentional  system.  We  have  started  with 
perhaps  the  most  primitive  and  basic  cues  as  to  object  structure:  namely,  that  features 
sharing  a  common,  rigid  motion  should  be  grouped  as  an  object  but  clearly  there  are  a 
variety  of  other  kinds  of  visual  information  which  would  be  important  as  well. 

Finally,  we  have  also  used  the  ERP  to  investigate  the  kind  of  representations  that 
occur  during  language  processing.  We  found  that  the  N400  reflects  a  level  of  semantic 
processing  that  is  common  for  both  pictures  and  words.  Thus  our  results  are  consistent 
with  the  claim  that  there  is  an  underlying  semantic  representation  that  is  neither 
perceptual  nor  linguistic. 
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